Autonomous driving system

ABSTRACT

An autonomous driving system includes: a Human Machine Interface; and a control device configured to control autonomous driving of a vehicle, present a first operation instruction to a driver of the vehicle during the autonomous driving, the first operation instruction requesting the driver to perform a first response operation performed in response to a first request or a first proposal by presenting the first request or the first proposal to the driver via the Human Machine Interface, and prohibit presenting a second operation instruction different from the first operation instruction until the first response operation is completed or until a timing at which the first response operation is predicted to be completed, the second operation instruction requesting the driver to perform a second response operation performed in response to a second request or a second proposal by presenting the second request or the second proposal.

INCORPORATION BY REFERENCE

The disclosure of Japanese Patent Application No. 2017-197031 filed onOct. 10, 2017 including the specification, drawings and abstract isincorporated herein by reference in its entirety.

BACKGROUND 1. Technical Field

The present disclosure relates to an autonomous driving system.

2. Description of Related Art

Published Japanese Translation of PCT Application No. 2013-544695(JP-A-2013-544695) discloses an autonomous driving system. Theautonomous driving system detects a zone, such as a lane junction and aconstruction site zone, where autonomous driving is difficult. When avehicle approaches such a zone where autonomous driving is difficult,the autonomous driving system requires the driver to control steering,acceleration, and deceleration.

SUMMARY

As exemplified in Published Japanese Translation of PCT Application No.2013-544695 (JP-A-2013-544695) described above, the autonomous drivingsystem sometimes sends an operation instruction to the driver duringautonomous driving. An operation instruction is an instruction thatsends a request or a proposal to the driver and that requests the driverto perform a response operation in response to the request or theproposal.

In some circumstances, a plurality of operation instructions may be sentconsecutively within a short period of time. However, the amount ofinformation and the number of operations that a human (driver) canprocess concurrently are limited. Therefore, when a plurality ofoperation instructions is sent consecutively in a short period of time,the driver feels impatient and uncomfortable. In addition, due to theshortage of time, there is a possibility that the driver cannot performthe response operation accurately. These inhibit the smooth traveling ofthe vehicle and reduce the reliability of the autonomous driving system.

For use in an autonomous driving system that sends an operationinstruction to the driver during autonomous driving, the presentdisclosure provides a technology that can reduce driver's impatience anddiscomfort.

An autonomous driving system includes: a Human Machine Interface; and

a control device configured to control autonomous driving of a vehicle,present a first operation instruction to a driver of the vehicle duringthe autonomous driving, the first operation instruction requesting thedriver to perform a first response operation performed in response to afirst request or a first proposal by presenting the first request or thefirst proposal to the driver via the Human Machine Interface, andprohibit presenting a second operation instruction different from thefirst operation instruction to the driver after the first operationinstruction is presented and until the first response operation iscompleted or until a timing at which the first response operation ispredicted to be completed, the second operation instruction requestingthe driver to perform a second response operation performed in responseto a second request or a second proposal by presenting the secondrequest or the second proposal to the driver via the Human MachineInterface.

According to the present disclosure, the sending of the next operationinstruction is prohibited until the response operation to an operationinstruction is completed or until the time at which at which theresponse operation is predicted to be completed. Therefore, the amountof information to be processed, and the number of operations to beexecuted, concurrently by the driver are reduced. As a result, thedriver's impatience and anxiety are reduced. In addition, the driver whois given enough time can easily perform the response operationaccurately. These allow a vehicle to travel smoothly and increase thereliability of the autonomous driving system.

BRIEF DESCRIPTION OF THE DRAWINGS

Features, advantages, and technical and industrial significance ofexemplary embodiments of the disclosure will be described below withreference to the accompanying drawings, in which like numerals denotelike elements, and wherein:

FIG. 1 is a conceptual diagram showing an autonomous driving systemaccording to an embodiment;

FIG. 2 is a conceptual diagram showing an example of a situation inwhich a plurality of operation instructions is sent consecutively withina short period of time;

FIG. 3 is a conceptual diagram showing another example of a situation inwhich a plurality of operation instructions is sent consecutively withina short period of time;

FIG. 4 is a conceptual diagram showing yet another example of asituation in which a plurality of operation instructions is sentconsecutively within a short period of time;

FIG. 5 is a conceptual diagram schematically showing the embodiment;

FIG. 6 is a block diagram showing a configuration example of anautonomous driving system according to the embodiment;

FIG. 7 is a block diagram showing an example of driving environmentinformation used in the autonomous driving system according to theembodiment;

FIG. 8 is a flowchart showing the operation instruction sendingprocessing performed by the autonomous driving system according to theembodiment;

FIG. 9 is a conceptual diagram showing the processing in step S30 in theoperation instruction sending processing according to the embodiment;

FIG. 10 is a conceptual diagram showing the processing in step S40 inthe operation instruction sending processing according to theembodiment;

FIG. 11 is a conceptual diagram showing a first example of thedetermination of a preceding operation instruction in the operationinstruction sending processing according to the embodiment;

FIG. 12 is a conceptual diagram showing a second example of thedetermination of a preceding operation instruction in the operationinstruction sending processing according to the embodiment;

FIG. 13 is a conceptual diagram showing a third example of thedetermination of a preceding operation instruction in the operationinstruction sending processing according to the embodiment;

FIG. 14 is a conceptual diagram showing a fourth example of thedetermination of a preceding operation instruction in the operationinstruction sending processing according to the embodiment; and

FIG. 15 is a conceptual diagram showing a fifth example of thedetermination of a preceding operation instruction in the operationinstruction sending processing according to the embodiment;

DETAILED DESCRIPTION OF EMBODIMENTS

An embodiment of the present disclosure will be described below withreference to the attached drawings.

FIG. 1 is a conceptual diagram showing an autonomous driving system 10according to an embodiment. The autonomous driving system 10 is mountedon a vehicle 1 for controlling the autonomous driving of the vehicle 1.For smooth traveling of the vehicle, the autonomous driving system 10sometimes sends an “operation instruction I” to the driver duringautonomous driving. The operation instruction I is an instruction thatmakes a request or a proposal to the driver and that requests the driverto perform a “response operation R” in response to the request or theproposal.

For example, the autonomous driving system 10 proposes a lane change(LC: Lane Chang) so that the vehicle arrives at the destination. Such alane change is required, for example, in a situation where there is alane branch or a lane merge. When a lane change is proposed, the driveris required to approve or refuse the lane change proposal (hereinafterreferred to as an “LC proposal”). That is, the “LC proposal” is anoperation instruction I, and the response operation R that is performedin response to the LC proposal is an “approval/refusal”.

As another example, the autonomous driving system 10 proposes passing apreceding vehicle. Passing can be thought of as one type of the LCproposal. When passing is proposed, the driver is required to approve orrefuse the passing proposal. That is, the “passing proposal” is anoperation instruction I, and the response operation R that is performedin response to the passing proposal is “approval/refusal”.

As yet another example, the autonomous driving system 10 requests thedriver to hold the steering wheel. Holding the steering wheel ishereinafter referred to as “steering wheel holding”. For example, whenthere is a sharp curve ahead of the vehicle 1 during autonomous driving,the autonomous driving system 10 requests the driver to hold thesteering wheel considering the possibility of lane departure. Inresponse to such a request, the driver holds the steering wheel. Thatis, the “steering wheel holding request” is an operation instruction I,and the response operation R that is performed in response to thesteering holding request is “steering wheel holding”.

As yet another example, the autonomous driving system 10 requests thedriver to start manual driving. Manual driving is required, for example,in a situation when the vehicle 1 arrives near the destination, when theautonomous driving permitted zone ends, or when there is an event whereautonomous driving is not suitable (for example, road work zone, complexterrain). When manual driving is requested, the driver performs themanual driving operation (for example, steering wheel holding, steeringoperation, accelerator operation, brake operation). That is, the “manualdriving request” is an operation instruction I, and the responseoperation R that is performed in response to the manual driving requestis a “manual driving operation”.

When the autonomous driving system 10 sends an operation instruction I,the driver is required to understand the contents of the operationinstruction I and, in addition, perform the response operation R.However, the amount of information and the number of operations that ahuman (driver) can process concurrently are limited. Therefore, when aplurality of operation instructions I is sent consecutively in a shortperiod of time, the driver cannot process them and may feel impatientand uncomfortable.

FIG. 2 is a conceptual diagram showing an example of the situation inwhich a plurality of operation instructions I is consecutively sent in ashort period of time. The vehicle 1 is traveling in lane L1, and thereis a lane branch ahead of the vehicle 1. To arrive at the destination,the autonomous driving system 10 plans to change the lane from lane L1to branch lane LB at position XA. In addition, at position XBimmediately after entering branch lane LB, manual driving is required.For example, there is a toll booth beyond position XB, or the autonomousdriving permitted zone (for example, an express way) ends at a positionahead of position XB. In this case, the autonomous driving system 10sends the “branching LC proposal” before position XA and sends the“manual driving request” before position XB.

FIG. 3 is a conceptual diagram showing another situation. The vehicle 1is traveling in a merging lane LM. The merging lane LM merges into laneL1, there is lane L2 adjacent to lane L1 and, in addition, branch laneLB branches from lane L2. In this situation, the autonomous drivingsystem 10 plans to change the lane from the merging lane LM to lane L1at position XA. In addition, the autonomous driving system 10 plans toenter branch lane LB to arrive at the destination. To enter branch laneLB, it is necessary to change the lane to lane L2 in advance. Therefore,at position XB immediately after entering lane L1, the autonomousdriving system 10 plans to change the lane from lane L1 to lane L2 inadvance. In this case, the autonomous driving system 10 sends the“merging LC proposal” before position XA, and the “pre-sending LCproposal” before position XB.

FIG. 4 is a conceptual diagram showing still another situation. There isa sharp curve ahead of the vehicle 1. The autonomous driving system 10requests the driver to hold the steering wheel considering thepossibility of lane departure. More specifically, the autonomous drivingsystem 10 sends the “steering wheel holding request” before the positionXA where the sharp curve starts. In addition, at position XB ahead ofposition XA, manual driving is required. For example, there is thedestination or a road work zone beyond position XB. In this case, theautonomous driving system 10 sends the “manual driving request” beforeposition XB.

As exemplified above, there are cases where a plurality of operationinstructions I is sent consecutively within a short period of time.Since the amount of information and the number of operations that ahuman (driver) can process concurrently are limited, the driver feelsimpatient and uncomfortable when a plurality of operation instructions Iis sent consecutively in a short period of time. In addition, due to theshortage of time, there is a possibility that the driver cannot performthe response operation R accurately. These inhibit the smooth travelingof the vehicle and reduce the reliability of the autonomous drivingsystem 10.

To solve such a problem, the autonomous driving system 10 according tothis embodiment positively adjusts the sending of operation instructionsI. More specifically, after an operation instruction I is sent and untilthe response operation R is completed or until the time comes at whichthe response operation R is expected to be completed, the autonomousdriving system 10 prohibits the sending of the next operationinstruction I.

FIG. 5 is a conceptual diagram schematically showing the embodimentdescribed above. The operation instruction time TI is the time at whichan operation instruction I is sent. The response operation completiontime TR is the time at which the response operation R that is performedin response to the operation instruction I is completed, or the time atwhich the response operation R is predicted to be completed. Theresponse operation period PR is a period from the operation instructiontime TI to the response operation completion time TR. During thisresponse operation period PR, it is prohibited to send the nextoperation instruction I to the driver.

According to this embodiment, the sending of the next operationinstruction I is prohibited as described above until the responseoperation R that is performed in response to an operation instruction Iis completed, or until the time at which the response operation R isexpected to be completed. Therefore, the amount of information to beprocessed, and the number of operations to be executed, concurrently bythe driver are reduced. As a result, the driver's impatience and anxietyare reduced. In addition, the driver who is given enough time canperform the response operation R accurately. These allow a vehicle totravel smoothly and to increase the reliability of the autonomousdriving system 10.

The configuration and processing of the autonomous driving system 10according to this embodiment will be described in more detail.

2. Configuration Example of the Autonomous Driving System

FIG. 6 is a block diagram showing a configuration example of theautonomous driving system 10 according to this embodiment. Theautonomous driving system 10 includes a Global Positioning System (GPS)receiver 20, a map database 30, sensors 40, a communication device 50, aHuman Machine Interface (HMI) unit 60, a response operation sensor 70, atraveling device 80, and a control device 100.

The GPS receiver 20 receives signals sent from a plurality of GPSsatellites and calculates the position and orientation of the vehicle 1based on the received signals.

The map database 30 records map information therein. The map informationincludes the information on the lane arrangement, lane attribute,autonomous driving permitted zone, location of facilities (for example,a toll booth), and so on.

The sensors 40 detect the surrounding situation of the vehicle 1 and thetraveling state of the vehicle 1. Examples of the sensors 40 includeLidar (LIDAR: Laser Imaging Detection and Ranging), a radar, a camera,and a vehicle speed sensor. Lidar uses light to detect a target aroundthe vehicle 1. The radar uses radio waves to detect a target around thevehicle 1. The camera captures the situation around the vehicle 1. Thevehicle speed sensor detects the speed of the vehicle 1.

The communication device 50 communicates with the outside of the vehicle1. For example, the communication device 50 performs V2I communication(road-vehicle communication) with the surrounding infrastructures. Thecommunication device 50 may perform V2V communication(vehicle-to-vehicle communication) with the surrounding vehicles. Thecommunication device 50 can also communicate with the management serverthat manages the autonomous driving service via a communication network.

The HMI unit 60 is an interface for providing information to the driverand for accepting information from the driver. More specifically, theHMI unit 60 has an input device and an output device. Examples of theinput device include a touch panel, a switch, and a microphone. Examplesof the output device include a display device and a speaker. The outputdevice is used for notifying an operation instruction I to the driver.The input device is used by the driver to enter the response operation R(in particular approval/refusal).

The response operation R performed by the driver is not limited to“approval/refusal”. The response operation R may be “steering wheelholding” or “manual driving request operation” in some cases. Theresponse operation sensor 70 is a sensor for detecting the responseoperation R other than “approval/refusal”. For example, the responseoperation sensor 70 includes a steering touch sensor for detectingwhether the driver is holding the steering wheel. The response operationsensor 70 may also include sensors for detecting the steering operation,the accelerator operation, and the brake operation, respectively.

The traveling device 80 includes a steering device, a driving device,and a braking device. The steering device steers the wheels. The drivingdevice is a power source that generates a driving force. Examples of thedriving device include an electric motor and an engine. The brakingdevice generates a braking force.

The control device 100 controls the autonomous driving of the vehicle 1.The control device 100 is a microcomputer including a processor 110 anda storage device 120. The control device 100 is also called anElectronic Control Unit (ECU). The processor 110 executes the controlprograms, stored in the storage device 120, to cause the control device100 to perform autonomous driving control.

In more detail, the control device 100 acquires information necessaryfor autonomous driving control. The information necessary for theautonomous driving control is hereinafter referred to as “drivingenvironment information 200”. The driving environment information 200,stored in the storage device 120, is read as necessary.

FIG. 7 shows an example of the driving environment information 200 inthis embodiment. The driving environment information 200 includesposition and orientation information 220, map information 230, sensordetection information 240, delivery information 250, and responseoperation information 260.

The position and orientation information 220 indicates the position andorientation of the vehicle 1. The control device 100 acquires theposition and orientation information 220 from the GPS receiver 20.

The map information 230 includes the information on the lanearrangement, lane attribute, autonomous driving permitted zone, locationof facilities (for example, a toll booth), and so on. The control device100 acquires the map information 230 around the vehicle 1 based on theposition and orientation information 220 and the map database 30. Thecontrol device 100 can collect information on lane merging, lanebranching, intersections, and lane curvatures based on the lanearrangement and the lane attributes indicated by the map information230.

The sensor detection information 240 is the information obtained fromthe detection result of the sensors 40. More specifically, the sensordetection information 240 includes the target information on the targetsaround the vehicle 1. Examples of the targets around the vehicle 1 aresurrounding vehicles, pedestrians, roadside objects, white lines, signs,and so on. In addition, the sensor detection information 240 includesthe vehicle speed detected by the vehicle speed sensor. The controldevice 100 acquires the sensor detection information 240 based on thedetection result of the sensors 40.

The delivery information 250 is the information obtained through thecommunication device 50. For example, the delivery information 250includes the road traffic information (congestion information, road workzone information, accident information, traffic regulation information,etc.) delivered from the infrastructure. The delivery information 250may include the information delivered from the management server thatmanages the autonomous driving service. The control device 100 acquiresthe delivery information 250 by communicating with the outside using thecommunication device 50.

The response operation information 260 is the information indicatingwhether the response operation R has been performed by the driver. Forexample, the control device 100 acquires the response operationinformation 260 on approval/refusal through the HMI unit 60. The controldevice 100 also acquires the response operation information 260 on theresponse operation R other than approval/refusal from the responseoperation sensor 70.

The control device 100 controls the autonomous driving of the vehicle 1based on the driving environment information 200 such as the onedescribed above. More specifically, the control device 100 generates atravel plan of the vehicle 1 based on the driving environmentinformation 200. Then, the control device 100 controls the travelingdevice 80 and causes the vehicle 1 to travel according to the travelplan.

In addition, the control device 100 plans the sending of an operationinstruction I as needed during autonomous driving. Then, the controldevice 100 sends (notifies) the operation instruction I to the driverusing the HMI unit 60. The driver performs the response operation R inresponse to the operation instruction I. The control device 100 confirmsthat the response operation R has been performed by the driver, based onthe response operation information 260. For example, if the operationinstruction I is an “LC proposal” and the “LC proposal” is approved bythe driver, the control device 100 controls the traveling device 80 tocause the vehicle to change the lane.

According to this embodiment, the control device 100 prohibits thesending the next operation instruction I after an operation instructionI is sent and until the response operation R is completed, or until theresponse operation R is expected to be completed. The processingperformed by the control device 100 to send an operation instruction Iis called the “operation instruction sending processing” in thedescription below. An example of the operation instruction sendingprocessing in this embodiment will be described in detail below.

3. Example of Operation Instruction Sending Processing

FIG. 8 is a flowchart showing the operation instruction sendingprocessing according to this embodiment. The processing flow shown inFIG. 8 is repeatedly executed for each predetermined cycle.

3-1. Step S10

The control device 100 detects an event related to an operationinstruction I based on the driving environment information 200. An eventrelated to an operation instruction I is an event for which the controldevice 100 plans to send an operation instruction I.

For example, events related to the “LC proposal” are lane branching,lane merging, and so on. These events can be detected based on the mapinformation 230.

As another example, events related to the “passing proposal” are alow-speed preceding vehicles and so on. A low-speed preceding vehiclecan be detected based on the sensor detection information 240 (targetinformation and vehicle speed information).

As a still another example, events related to the “steering wheelholding request” are a sharp curve ahead of the vehicle 1 and so on. Asharp curve can be detected based on the map information 230 (lanearrangement information).

As a still another example, events related to the “manual drivingrequest” are a pre-set destination, a toll booth, and the end of anautonomous driving permitted zone, and so on. These events can bedetected based on the map information 230. In addition, an event that isdifficult to be handled during autonomous driving, such as a road workzone, a traffic congestion zone, and a complex terrain, can be thoughtof as an event related to the “manual driving request”. A road work zoneand a traffic congestion zone can be recognized based on the deliveryinformation 250. A complex terrain is recognizable based on the mapinformation 230.

If an event related to an operation instruction I is detected (step S10;Yes), the processing proceeds to step S20. Otherwise (step S10; No), theprocessing proceeds to step S50.

3-2. Step S20

The control device 100 plans an operation instruction I according to theevent detected in step S10. More specifically, the control device 100determines the type of the operation instruction I according to thedetected event and, in addition, plans an operation instruction time TIat which the operation instruction I is to be sent. The operationinstruction time TI is earlier than the time at which the vehicle 1 willarrive at the detected event. The control device 100 calculates the timeat which the vehicle 1 will arrive at the detected event, based on theposition and orientation information 220, the map information 230, andthe sensor detection information 240 (vehicle speed information) and, asnecessary, plans the operation instruction time TI. After that, theprocessing proceeds to step S30.

3-3. Step S30

The control device 100 determines whether the response operation periodPR (see FIG. 5) for the operation instruction I, planned in step S20,overlaps the response operation period PR for an existing operationinstruction I that has already been planned.

FIG. 9 is a conceptual diagram showing the processing in step S30. Inthe description below, consider the case where there are two operationinstructions, “first operation instruction I1” and “second operationinstruction I2”, as the candidate for the operation instruction I.

The first operation instruction time TI1 is the time at which the firstoperation instruction I1 is sent. The first response operation R1 is theresponse operation R that is performed in response to the firstoperation instruction I1. The first response operation completion timeTR1, a time later than the first operation instruction time TI1 by thefirst predicted time ΔT1, is the time at which the first responseoperation R1 is predicted to be completed. The first response operationperiod PR1 is a period from the first operation instruction time TI1 tothe first response operation completion time TR1.

The second operation instruction time TI2 is the time at which thesecond operation instruction I12 is sent. The second response operationR2 is the response operation R that is performed in response to thesecond operation instruction I2. The second response operationcompletion time TR2, a time later than the second operation instructiontime TI2 by the second predicted time ΔT2, is the time at which thesecond response operation R2 is predicted to be completed. The secondresponse operation period PR2 is a period from the second operationinstruction time TI2 to the second response operation completion timeTR2.

The predicted time ΔT from the operation instruction time TI to theresponse operation completion time TR is predetermined for each type ofthe operation instruction I. For example, when the operation instructionI is the “LC proposal” or the “passing proposal”, the predicted time ΔTis set to 8 seconds. When the operation instruction I is the “steeringwheel holding request”, the predicted time ΔT is set to 6 seconds. Whenthe operation instruction I is the “manual driving request”, thepredicted time ΔT is set to 6 seconds. The setting information on thepredicted times ΔT is pre-stored in the storage device 120. The controldevice 100 can acquire the predicted time ΔT according to an operationinstruction I by referring to the setting information.

The control device 100 predicts the first response operation completiontime TR1, that is, the first response operation period PR1, by addingthe first predicted time ΔT1 to the first operation instruction time TI1planned in step S20. Similarly, the control device 100 predicts thesecond response operation completion time TR2, that is, the secondresponse operation period PR2, by adding the second predicted time ΔT2to the second operation instruction time TI2 planned in step S20.

Then, the control device 100 determines whether the predicted firstresponse operation period PR1 and the predicted second responseoperation period PR2 overlap. In the example shown in FIG. 9, the firstresponse operation period PR1 and the second response operation periodPR2 overlap. If the first response operation period PR1 and the secondresponse operation period PR2 overlap (step S30; Yes), the processingproceeds to step S40. Otherwise (step S30; No), the processing proceedsto step S50.

3-4. Step S40

The control device 100 adjusts at least one of the first operationinstruction time TI1 and the second operation instruction time TI2 sothat the first response operation period PR1 and the second responseoperation period PR2 do not overlap.

If neither the first operation instruction I1 nor the second operationinstruction I2 has been sent, it is first necessary to determine whichone of the first operation instruction I1 and the second operationinstruction I2 is to be sent first. One of the first operationinstruction I1 and the second operation instruction I2, whichever issent first, is hereinafter called a “preceding operation instruction IA”and the other is called a “following operation instruction IB”. Thecontrol device 100 determines the preceding operation instruction IA,based on the combination of the first operation instruction I1 and thesecond operation instruction I2. As a method for determining whichoperation instruction, either first operation instruction I1 or secondoperation instruction I2, is the preceding operation instruction IA,various examples can be considered as will be described later.

FIG. 10 is a conceptual diagram showing the processing in step S40. Thepreceding operation instruction time TIA is the time at which thepreceding operation instruction IA is sent. The preceding responseoperation RA is the response operation R that is performed in responseto the preceding operation instruction IA. The preceding responseoperation completion time TRA is the time at which the precedingresponse operation RA is predicted to be completed. The precedingresponse operation period PRA is a period from the preceding operationinstruction time TIA to the preceding response operation completion timeTRA.

The following operation instruction time TIB is time at which thefollowing operation instruction IB is sent. The following responseoperation RB is the response operation R that is performed in responseto the following operation instruction IB. The following responseoperation completion time TRB is the time at which the followingresponse operation RB is predicted to be completed. The followingresponse operation period PRB is a period from the following operationinstruction time TIB to the following response operation completion timeTRB.

The control device 100 adjusts at least one of the preceding operationinstruction time TIA and the following operation instruction time TIB sothat the preceding response operation period PRA and the followingresponse operation period PRB do not overlap. For example, the controldevice 100 sets the preceding operation instruction time TIA to a timeearlier than the planned time. Alternatively, the control device 100sets the following operation instruction time TIB to a time later thanthe planned time.

If the following operation instruction time TIB is too late, there is apossibility that the vehicle 1 will arrive at an event related to thefollowing operation instruction IB before the following operationinstruction IB is sent. That is, the following operation instruction IBmay be too late (meaningless). In that sense, it is safer to set thepreceding operation instruction time TIA to an earlier time rather thanto set the following operation instruction time TIB to a later time.

In the situation where one of the first operation instruction I1 and thesecond operation instruction I2 has already been sent and the controldevice 100 is waiting for the response operation R, the already sentoperation instruction I becomes the preceding operation instruction IAand the other instruction becomes the following operation instructionIB. In such a case, the control device 100 sets the following operationinstruction time TIB to a later time so that the preceding responseoperation period PRA and the following response operation period PRB donot overlap.

After step S40, the processing proceeds to step S50.

3-5. Steps S50 and S60

The control device 100 determines whether the “instruction sendingcondition” is satisfied (step S50). The instruction sending condition isa condition under which an operation instruction I, planned by thecontrol device 100, is actually sent to the driver. If the instructionsending condition is satisfied (step S50; Yes), the control device 100sends the operation instruction I to the driver via the HMI unit 60(step S60). If the instruction sending condition is not satisfied (stepS50; No), the current processing cycle ends and the processing returnsto step S10.

Typically, the instruction sending condition is the condition that the“operation instruction time TI has come”. For example, in the exampleshown in FIG. 10, when the preceding operation instruction time TIA hascome, the control device 100 sends the preceding operation instructionIA. After that, when the following operation instruction time TIB hascome, the control device 100 sends the following operation instructionIB.

As another example, the instruction sending condition for the followingoperation instruction IB may be the condition that the “precedingresponse operation RA has completed”. The control device 100 can confirmthe completion of the preceding response operation RA based on theresponse operation information 260. If the completion of the precedingresponse operation RA cannot be confirmed when the following operationinstruction time TIB has come, the control device 100 sets the followingoperation instruction time TIB to a later time. Conversely, if thefollowing operation instruction time TIB has not yet come when thecompletion of the preceding response operation RA can be confirmed, thecontrol device 100 may set the following operation instruction time TIBto an early time. In any case, the control device 100 sends thefollowing operation instruction IB after confirming that the precedingresponse operation RA has been completed. In this way, by finelyadjusting the following operation instruction time TIB through themonitoring of the actual preceding response operation RA, the accuracyis improved than when the following operation instruction time TIB thatis based only on the predicted value is used.

4. Various Examples for Determining the Preceding Operation Instruction

In step S40 described above, the control device 100 determines the firstoperation instruction I1 or the second operation instruction I2,whichever is sent first, as the “preceding operation instruction IA”. Atthis time, the control device 100 determines the preceding operationinstruction IA, based on the combination of the first operationinstruction I1 and the second operation instruction I2. There arevarious examples for determining the preceding operation instruction IA.

4-1. First Example

FIG. 11 is a conceptual diagram showing a first example for determiningthe preceding operation instruction IA. FIG. 11 shows the same situationas that in FIG. 2 described above. That is, there are two operationinstructions I, “branching LC proposal” and “manual driving request”, asthe candidate for the operation instruction I. The “branching LCproposal” is an operation instruction I related to the event “lanebranching”. On the other hand, the “manual driving request” is anoperation instruction I related to an event in which manual driving isrequired (example: there is a toll booth ahead of the vehicle,autonomous driving permitted zone is ended). In the situation shown inFIG. 11, the event “lane branching”, one of the two events, is nearer tothe current position of the vehicle 1. Therefore, the control device 100determines the “branching LC proposal” as the preceding operationinstruction IA.

4-2. Second Example

FIG. 12 is a conceptual diagram showing a second example for determiningthe preceding operation instruction IA. FIG. 12 shows the same situationas that in FIG. 3 described above. That is, there are two operationinstructions I, “merging LC proposal” and “pre-sending LC proposal”, asthe candidate for the operation instruction I. The “merging LC proposal”is an operation instruction I related to the event “lane merging”. Onthe other hand, the “pre-sending LC proposal” is an operationinstruction I related to the event “lane branching”. In the situationshown in FIG. 12, the event “lane merging”, one of the two events, isnearer to the current position of the vehicle 1. Therefore, the controldevice 100 determines the “merging LC proposal” as the precedingoperation instruction IA.

4-3. Third Example

FIG. 13 is a conceptual diagram showing a third example for determiningthe preceding operation instruction IA. FIG. 13 shows the same situationas that in FIG. 4 described above. That is, there are two operationinstructions I, “steering wheel holding request” and “manual drivingrequest”, as the candidate for the operation instruction I. The“steering wheel holding request” is an operation instruction I relatedto the event “sharp curve”. On the other hand, the “manual drivingrequest” is an operation instruction I related to the event in whichmanual driving is required (for example: the vehicle is approaching thedestination or a road work zone). In the situation shown in FIG. 13, theevent “sharp curve”, one of the two events, is nearer to the currentposition of the vehicle 1. Therefore, the control device 100 determinesthe “steering wheel holding request” as the preceding operationinstruction IA.

The above first to third examples are generalized as follows. The firstoperation instruction I1 is related to the first event, and the secondoperation instruction I2 is related to the second event. One of thefirst event and the second event, whichever is nearer to the currentposition of the vehicle 1, is a “preceding event”. In this case, thecontrol device 100 determines an operation instruction I that is one ofthe first operation instruction I1 and the second operation instructionI2 and that is related to the preceding event as the preceding operationinstruction IA. Since the operation instructions I are sent in the orderin which the events are detected, it is easy for the driver tounderstand the flow of driving.

4-4. Fourth Example

FIG. 14 is a conceptual diagram showing a fourth example for determiningthe preceding operation instruction IA. The vehicle 1 is traveling inthe right side lane L2. The control device 100 plans to send a“keep-left LC proposal” that proposes to change the lane to the leftlane L1 to keep to the left. In addition, since there is a sharp curveahead of the vehicle 1, the control device 100 plans to send the“steering wheel holding request” before position XA where the sharpcurve starts.

In this situation, the lane change for keeping to the left, which is notso urgent, may be performed at any time (for example, at position XBafter the sharp curve ends). That is, even if the lane change forkeeping to the left is delayed to some extent, there is no particulareffect on vehicle traveling. On the other hand, from the viewpoint ofvehicle traveling, it is desired to complete “steering wheel holding”before position XA where the sharp curve starts. That is, the priority(importance) of “steering wheel holding” is high, whereas the priorityof lane change for keeping to the left is not so high and its degree offreedom is high. Therefore, the control device 100 determines the“steering wheel holding request” as the preceding operation instructionIA, and the “keep-left LC proposal” as the following operationinstruction IB.

The fourth example is generalized as follows. An operation instruction Ithat requires that the response operation R be performed before thevehicle 1 arrives at a specific point is hereinafter referred to as a“priority operation instruction IP”. Examples of priority operationinstructions IP include not only the “steering wheel holding request”described above but also the “manual driving request”, “branching LCproposal”, and “merging LC proposal”. From the viewpoint of vehicletraveling, safety, and the arrival at the destination, the priority ofthe priority operation instruction IP is high. On the other hand, theabove-mentioned “keep-left LC proposal” and “passing proposal” are not apriority operation instruction IP and have a higher degree of freedom.If one of the first operation instruction I1 and the second operationinstruction I2 is a priority operation instruction IP and the other isnot, the control device 100 determines the priority operationinstruction IP as the preceding operation instruction IA. This makes itpossible to reduce the amount of information to be processed, and thenumber of operations to be executed, by the driver concurrently while atthe same time sending a high-priority operation instruction I withoutdelay.

4-5. Fifth Example

FIG. 15 is a conceptual diagram showing a fifth example for determiningthe preceding operation instruction IA. The fifth example also relatesto the priority operation instruction IP.

The vehicle 1 is traveling in lane L1. In the same lane L1 and ahead ofthe vehicle 1, there is a low-speed preceding vehicle 2. The controldevice 100 detects the low-speed preceding vehicle 2 based on the sensordetection information 240. Then, the control device 100 plans to startthe passing operation at position XA and plans to send the “passingproposal” before position XA. In addition, at position XB beyondposition XA, manual driving is required. For example, there is thedestination or a road work zone beyond position XB. Therefore, thecontrol device 100 plans to send the “manual driving request” beforeposition XB.

The manual driving request is a priority operation instruction IP, andthe passing proposal is not. Therefore, the control device 100determines the manual driving request as the preceding operationinstruction IA, and the passing proposal as the following operationinstruction IB. This makes it possible to send a high priority manualdriving request without delay.

When the driver starts manual driving in response to a manual drivingrequest, the result is that the “passing proposal” is not sent from theautonomous driving system 10 (control device 100). Even in this case,the next operation instruction is prohibited until the responseoperation R, which is performed in response to the operation instructionI, is completed. That is, this makes it possible to reduce the amount ofinformation to be processed, and the number of operations to beexecuted, by the driver concurrently and, as a result, reduces thedriver's impatience and anxiety.

4-6. Sixth Example

As described above, if one of the first operation instruction I1 and thesecond operation instruction I2 is a priority operation instruction IPand the other is not, the control device 100 determines the priorityoperation instruction IP as the preceding operation instruction IA. Atthis time, the control device 100 may discard the operation instructionthat is not a priority operation instruction IP. Even in this case, thenext operation instruction is prohibited until the response operation R,which is performed in response to the operation instruction I, iscompleted. That is, this makes it possible to reduce the amount ofinformation to be processed, and the number of operations to beexecuted, by the driver concurrently and, as a result, reduces thedriver's impatience and anxiety.

What is claimed is:
 1. An autonomous driving system comprising: a HumanMachine Interface; and a control device configured to control autonomousdriving of a vehicle, present a first operation instruction to a driverof the vehicle during the autonomous driving, the first operationinstruction requesting the driver to perform a first response operationperformed in response to a first request or a first proposal bypresenting the first request or the first proposal to the driver via theHuman Machine Interface, prohibit presenting a second operationinstruction different from the first operation instruction to the driverafter the first operation instruction is presented and until the firstresponse operation is completed or until a timing at which the firstresponse operation is predicted to be completed, the second operationinstruction requesting the driver to perform a second response operationperformed in response to a second request or a second proposal bypresenting the second request or the second proposal to the driver viathe Human Machine Interface, plan a first operation instruction timingand a second operation instruction timing, the first operationinstruction timing being a timing at which the first operationinstruction is to be presented, the second operation instruction timingbeing a timing at which the second operation instruction is to bepresented, predict a first response operation period and a secondresponse operation period based on the first operation instructiontiming and the second operation instruction timing the first responseoperation period being a period from the first operation instructiontiming to a first response operation completion timing at which thefirst response operation is predicted to be completed, the secondresponse operation period being a period from the second operationinstruction timing to a second response operation completion timing atwhich the second response operation is predicted to be completed, adjustat least one of the first operation instruction timing and the secondoperation instruction timing such that the first response operationperiod and the second response operation period do not overlap, when thefirst response operation period and the second response operation periodare predicted to overlap, determine, as a preceding operationinstruction, one of the first operation instruction and the secondoperation instruction whichever is to be presented first, and adjust atleast one of the first operation instruction timing and the secondoperation instruction timing to set a time of the preceding operationinstruction to a time earlier than a planned time.
 2. The autonomousdriving system according to claim 1, wherein the first operationinstruction is an operation instruction related to a first event and thesecond operation instruction is an operation instruction related to asecond event, and the control device is configured to determine, as apreceding event, one of the first event and the second event whicheverexists nearer to a current position of the vehicle, and determine anoperation instruction related to the preceding event, as the precedingoperation instruction, among the first operation instruction and thesecond operation instruction.
 3. The autonomous driving system accordingto claim 1, wherein a priority operation instruction is an operationinstruction for which a response operation needs to be performed beforethe vehicle arrives at a particular position, and the control device isconfigured to determine the priority operation instruction as thepreceding operation instruction when one of the first operationinstruction and the second operation instruction is the priorityoperation instruction and when the other is not the priority operationinstruction.
 4. The autonomous driving system according to claim 1,wherein a following operation instruction is an operation instructionthat is one of the first operation instruction and the second operationinstruction and is not the preceding operation instruction and apreceding response operation is the response operation that is performedin response to the preceding operation instruction and the controldevice is configured to present the following operation instruction tothe driver after the control device confirms that the preceding responseoperation is completed.
 5. The autonomous driving system according toclaim 1, wherein the Human Machine Interface is configured to accept thefirst response operation and the second response operation, and thecontrol device is configured to present the first operation instructionand the second operation instruction to the driver via the Human MachineInterface, acquire response operation information about the firstresponse operation via the Human Machine Interface, and determine thatthe first response operation is completed, based on the responseoperation information.
 6. The autonomous driving system according toclaim 5, wherein the response operation information includes informationabout whether the driver has approved the first request or the firstproposal.